Distributed Information Retrieval Based on Hierarchical Semantic Overlay Network

One fundamental problem that confronts information retrieval is to efficiently support query with higher accuracy and less logic hops. This paper presents HSPIR (Hierarchical Semantic P2P-based Information Retrieval) that distributes document indices through the P2P network hierarchically based on documents semantics generated by Latent Semantic Indexing (LSI) [1]. HSPIR uses CAN [2] and Range Addressable network organize [3] nodes into a hierarchical overlay network. Comparing with other P2P search techniques [4, 5] those are based on simple keyword matching, HSPIR has better accuracy for it considers the advanced relevance among documents. We use Agglomerative Information Bottleneck (AIB) [6] to cluster documents and train Directed Acyclic Graph Support Vector Machines (DAGSVM) based on these clustered documents. Owning to the hierarchical overlay network, the average number of logical hops per query is smaller than other flat architectures.